Learning to Rank with Supplementary Data

This paper is concerned with a new task of ranking, referred to as “supplementary data assisted ranking”, or “supplementary ranking” for short. Different from conventional ranking, in the new task, each query is associated with two sets of objects: the target objects that are to be ranked, and the supplementary objects whose orders are not of our interest. Existing learning to rank approaches (either supervised or semi-supervised) cannot well handle the new task, because they ignore the supplementary data in either training, test, or both. In this paper, we propose a general approach for the task, in which the ranking model consists of two parts. The first part is based on the matching between a target object and the query (which is similar to that in conventional approaches). The second part depends on the relationship between target objects and supplementary objects. The new ranking model is learned by minimizing a certain loss function on the training data. We call this approach “supplementary learning to rank”. As a showcase of the approach, we develop two Boosting-style algorithms. In these algorithms, we leverage the supplementary objects in the definition of weak rankers for the second part of the ranking model, and specify the relationship between target and supplementary objects as pairwise preference. Experimental results on both public and large-scale commercial datasets demonstrate the effectiveness of the proposed algorithms.